1. Field of the Invention
The present invention generally relates to data processing systems and more particularly relates to such systems that analyze survey data.
2. Description of the Prior Art
Many businesses have adopted the concept of Total Quality Management (TQM) to help improve profitability, ensure sustained customer loyalty, improve quality of products, etc. Originally developed by E. Deming and others, TQM is now widely accepted in many business circles throughout the world. A cornerstone of most total quality management (TQM) systems is the periodic measurement of identified parameters that relate to various aspects of the business. Through these measurements, managers can identify the areas within the organization that need improvement. Because the measurements are made on a xe2x80x9cperiodicxe2x80x9d basis, managers can gauge the effects of various changes made within the organization over time. The goal of most TQM systems is to make continuous improvement within the organization.
Businesses often generate and manage an array of measurements for purposes of analyzing the performance of individual operations in relationship to productivity, resource utilization, profitability, etc. One of the most important measurements in many TQM systems is that of customer satisfaction. To measure customer satisfaction, periodic customer feedback is required, usually obtained through customer surveys.
In many surveys, customers are asked a number of survey questions that are related to various products and/or services provided by a business. For many questions, the customers can respond both with a satisfaction rating and an importance rating. By analyzing the answers to these questions, managers can obtain insights into many areas, including customer loyalty, overall satisfaction, potential weakness, etc.
It is not uncommon for businesses to conduct their own customer surveys. However, it is increasingly popular to outsource this function to outside vendors who specialize in conducting customer surveys. Today, there are a number of vendors who conduct customer surveys, and provide a survey database to the subscribing business. One such company is Prognostics Inc., located in Menlo Park, Calif. An advantage of using such a vendor is that customer survey data for other companies, including competing companies, can often be obtained. This enables a business to perform comparisons between itself and its competitors.
Some of the vendors of customer surveys provide the survey data in electronic form, and often provide a software program for accessing the survey database. The software programs facilitate the generation of reports or the like, which can then be used by management. Prognostics is one such vendor. Prognostics provides survey data in a proprietary electronic format, and provides a software program called the Research Analysis Program (RAP) which can access the survey data. RAP can read the survey database, perform data requests, and provide a number of reports.
A limitation of many of the survey analysis programs is that the survey results may mislead the user. For example, survey results may be based on a statistically insignificant sample size, thereby misleading the user. Similarly, survey results may be based on data elements that skew the results in an undesirable way. Often, the user is unaware that the survey results have these deficiencies, and may base important business decisions on the misrepresentative survey results.
Misrepresentative results can often be traced to portions of the survey database that are either under-represented, or otherwise different from the user""s expectations. The underline assumption when using a typical survey analysis program is that the survey database is fully represented in all respects, and that all data elements fall within the user""s expectations. It has been found that this is often not the case.
For example, it is known that many survey analysis programs allow the user to make qualified user requests that select and operate only on a portion of the database. Using such a qualified request, a user may request the overall satisfaction for those respondents that are in a particular industry sector, that are located in a particular geographic region, and that use a particular product. The number of respondents represented in the survey database that meet all of these criteria may be relatively small. Thus, in this case, the corresponding results may be based on a statistically insignificant sample size. The user may not recognize this, and may make important business decisions based upon the misrepresentative results.
In another example, the survey database may include data elements that skew the results in an undesirable way. For example, it is known that many questions in a customer survey may solicit two answers, such as an overall importance question and an overall satisfaction question. Those responses that indicate a high degree of satisfaction but a low degree of importance may skew the results provided by an overall satisfaction request. In addition, some respondents may only answer one of the two questions, such as the satisfaction question but not the importance question. This may also skew the results of certain user requests.
The present invention overcomes many of the disadvantages of the prior art by providing a method and apparatus for warning the user of potential limitations of a database request and/or the results provided thereby. Preferably, the identified limitations are provided in a number of caveats. The caveats may warn the user that a survey request and/or survey result may be improper, invalid or otherwise deficient in some way. This may prevent the user from basing important business decisions on misrepresentative results.
In accordance with the present invention, these and other advantages are preferably accomplished by using a rules-based expert system for forming and executing requests to a survey database. In a rules-based expert system, a number of rules are provided wherein the rules contain much of the xe2x80x9cknowledgexe2x80x9d of the experts, and the insights of xe2x80x9cdecision makersxe2x80x9d. This allows xe2x80x9cnon-expertxe2x80x9d users to perform xe2x80x9cexpertxe2x80x9d analysis of client satisfaction data in an accurate and repeatable fashion. This may make data analysis of a survey database faster, more reliable, and cheaper than in the prior art. Moreover, additional request types and/or caveats can be added to the system by simply adding additional rules, and linking the additional rules into the system.
Preferably, the rules-based expert system uses a number of predefined rules to process a request. Each rule is similar to a conditional statement in a conventional computer program (such as C++). However, the rules-based expert system is equipped with an inference engine, which is a special program for managing rules and applying them as appropriate. By contrast, a conventional program must indicate explicitly when a given conditional statement should be applied.
In one embodiment of the present invention, a system is provided that has an interface module for accepting a request from a user, and a knowledge module that communicates with the interface module and is capable of accessing the survey database. The knowledge module processes the request and provides a number of data requests to the survey database. This is accomplished, at least in part, by executing selected ones of a number of predefined rules. An inference engine selects which of the number of predefined rules are actually applied for a particular request. The knowledge module then receives the request data elements from the survey database, and derives the desired result therefrom. This is also preferably accomplished, at least in part, by executing selected ones of a number of predefined rules. The desired result is then provided to the interface module for viewing by the user.
It is contemplated that the knowledge module may execute a number of request caveat rules to identify any request caveats for a particular user request. Preferably, a request caveat may indicate if the request is a valid request, as determined by the xe2x80x9cexpertsxe2x80x9d of a business organization. This knowledge may be included in the request caveat rules. For example, the xe2x80x9cexpertsxe2x80x9d of a business organization may determine that a comparison of customer satisfaction between a mainframe computer system and a personal computer system does not provide any valid or useful information. In this instance, this request may be labeled as invalid or improper. Because the rules in a rules-based expert system include knowledge provided by the xe2x80x9cexpertsxe2x80x9d, the rules may also provide a number of reasons why the user request is determined to be improper or invalid.
In a preferred embodiment, the user request is formed by selecting a combination of available selections via a graphical user interface. The present invention determines if the user request is an appropriate or proper request by determining if the combination of selections made by the user corresponds to one of a number of predetermined appropriate combination of selections. Even though the user request may be in a proper form (e.g. proper syntax), the present invention may identify the request as improper if it is determined that it will provide misrepresentative or otherwise improper results. Despite any request caveats that are provided, it is contemplated that the user may still elect to process the user request, and view the results as desired.
A second illustrative type of caveat is a result caveat. A result caveat preferably provides an indication of confidence in the result. A low indication of confidence may indicate that the results should be viewed with caution by the user. Preferably, the knowledge module identifies those portions of the survey database that are potentially problematic, including those portions that are under-represented or otherwise capable of skewing the results in an undesirable way. To accomplish this, the knowledge module may execute a number of result caveat rules. The result caveat rules include the knowledge of the xe2x80x9cexpertsxe2x80x9d, and operates on the data elements received from the survey database.
In one example, the result caveat rules may determine if the data elements retrieved by the knowledge module are sufficient to derive a statistically significant response to the user request. That is, if there are insufficient data elements in the survey database to produce a statistically significant result for a particular user request, a result caveat may be provided, indicating that the results should be viewed with caution. The result caveats may also identify a group of selected data elements within the survey database that caused the result to be statistically insignificant.
In another example, the result caveats may provide an explanation of the result. The explanation may identify the algorithm used to produce the result, and if any irregularities occurred during the analysis, etc. The irregularities may include the identification of portions of the survey database that differ from a predetermined expectation. As indicated above, a user typically expects that the survey database is fully represented, has a certain distribution of responses for various survey questions, etc. The result caveats may identify those irregularities, and provide an explanation thereof.